The Google Challenge: Video Genre Classification

Video files are large and usually take long time to view. Manual browsing and organizing a big collection of video files is tedious and impractical. The goal of this project is to develop an automatic method for video classification based on their genre,. Take movie as examples, the program will classify movies into action movies, comedy movies, tragedy movies, etc. In other words, our algorithm can 'understand' the 'plot' of a movie based on its image content without using metadata or other manual annotations. Conventional photometric features, e.g. color histogram and SIFT bear little semantic meaning. So we propose other ways that can capture this semantic meaning. We argue that facial expression of characters and interactions are very much related to the story line. A graph of such interactions will be a good visualization of movies. To begin with, we use face detector to detect all the characters. Clustering on the detection result will reveal the main characters of the movies. A supervised clustering technique has been tested on face images from movie, and yield promising results. Facial expressions of the main characters are recognized to further estimate the emotional variations. We have shown that concepts such as happy, angry and asad can be learnt using collective resources like Flickr and Google. Inspired by social network visualization techniques, a novel graphical method is presented to extract and visualize genre of a movie.